Search for: All records

Abstract The spatiotemporal evolution of marine heatwaves (MHWs) is explored using a tracking algorithm called Ocetrac that provides the objective characterization of MHW spatiotemporal evolution. Candidate MHW grid points are defined in detrended gridded sea temperature data using a seasonally varying temperature threshold. Identified MHW points are collected into spatially distinct objects using edge detection with weak sensitivity to edge detection and size percentile threshold criteria at each time step. Ocetrac then uses 3D connectivity to determine if these objects are part of the same event, but Ocetrac only defines the full MHW event after all time steps have been processed, limiting its use in predictability studies. Here, Ocetrac is applied to monthly satellite sea surface temperature data from September 1981 through January 2021. The resulting MHWs are characterized by their intensity, duration, and total area covered. The global analysis shows that MHWs in the Gulf of Maine and Mediterranean Sea are spatially isolated, while major MHWs in the Pacific and Indian Oceans are connected in space and time. The largest and most long-lasting MHW using this method lasts for 60 months from November 2013 to October 2018, encompassing previously identified MHW events including those in the northeast Pacific (2014–15), the Tasman Sea (2015–16, 2017–18), and the Great Barrier Reef (2016). Significance StatementThis study introduces Ocetrac, a method to track the spatiotemporal evolution of marine heatwaves (MHWs). It is applied to satellite sea surface temperature data from 1981 to 2021. The method objectively identifies and tracks MHWs in space and time while allowing for splitting and merging. The resulting MHWs are characterized by intensity, duration, and total area covered. Marine heatwaves can have significant ecological consequences, including biodiversity loss and mortality, geographical shifts, and range reductions in marine species and community structure changes when physiological thresholds are exceeded. This results in both ecological and economic impacts. Ocetrac provides a method of tracking the space and time evolution of MHWs that can provide a visualization that demonstrates the global impact of these events. 

Computer simulations of the Earth’s climate and weather generate huge amounts of data. These data are often persisted on HPC systems or in the cloud across multiple data assets of a variety of formats (netCDF, zarr, etc...). Finding, investigating, loading these data assets into compute-ready data containers costs time and effort. The data user needs to know what data sets are available, the attributes describing each data set, before loading a specific data set and analyzing it. In this notebook, we demonstrate the integration of data discovery tools such as intake and intake-esm (an intake plugin) with data stored in cloud optimized formats (zarr). We highlight (1) how these tools provide transparent access to local and remote catalogs and data, (2) the API for exploring arbitrary metadata associated with data, loading data sets into data array containers. We also showcase the Pangeo catalog, an open source project to enumerate and organize cloud optimized climate data stored across a variety of providers, and a place where several intake-esm collections are now publicly available. We use one of these public collections as an example to show how an end user would explore and interact with the data, and conclude with a short overview of the catalog's online presence. 

As more analysis-ready datasets are provided on the cloud, we need to consider how researchers access data. To maximize performance and minimize costs, we move the analysis to the data. This notebook demonstrates a Pangeo deployment connected to multiple Dask Gateways to enable analysis, regardless of where the data is stored. Public clouds are partitioned into regions, a geographic location with a cluster of data centers. A dataset like the National Water Model Short-Range Forecast is provided in a single region of some cloud provider (e.g. AWS’s us-east-1). To analyze that dataset efficiently, we do the analysis in the same region as the dataset. That’s especially true for very large datasets. Making local “dark replicas” of the datasets is slow and expensive. In this notebook we demonstrate a few open source tools to compute “close” to cloud data. We use Intake as a data catalog, to discover the datasets we have available and load them as an xarray Dataset. With xarray, we’re able to write the necessary transformations, filtering, and reductions that compose our analysis. To process the large amounts of data in parallel, we use Dask. Behind the scenes, we’ve configured this Pangeo deployment with multiple Dask Gateways, which provide a secure, multi-tenant server for managing Dask clusters. Each Gateway is provisioned with the necessary permissions to access the data. By placing compute (the Dask workers) in the same region as the dataset, we achieve the highest performance: these worker machines are physically close to the machines storing the data and have the highest bandwidth. We minimize cost by avoiding egress costs: fees charged to the data provider when data leaves a cloud region. 

Modern climate projections lack adequate spatial and temporal resolution due to computational constraints. A consequence is inaccurate and imprecise predictions of critical processes such as storms. Hybrid methods that combine physics with machine learning (ML) have introduced a new generation of higher fidelity climate simulators that can sidestep Moore's Law by outsourcing compute-hungry, short, high-resolution simulations to ML emulators. However, this hybrid ML-physics simulation approach requires domain-specific treatment and has been inaccessible to ML experts because of lack of training data and relevant, easy-to-use workflows. We present ClimSim, the largest-ever dataset designed for hybrid ML-physics research. It comprises multi-scale climate simulations, developed by a consortium of climate scientists and ML researchers. It consists of 5.7 billion pairs of multivariate input and output vectors that isolate the influence of locally-nested, high-resolution, high-fidelity physics on a host climate simulator's macro-scale physical state.The dataset is global in coverage, spans multiple years at high sampling frequency, and is designed such that resulting emulators are compatible with downstream coupling into operational climate simulators. We implement a range of deterministic and stochastic regression baselines to highlight the ML challenges and their scoring. The data (https://huggingface.co/datasets/LEAP/ClimSim_high-res) and code (https://leap-stc.github.io/ClimSim) are released openly to support the development of hybrid ML-physics and high-fidelity climate simulations for the benefit of science and society. 

Abstract We describe a new way to apply a spatial filter to gridded data from models or observations, focusing on low‐pass filters. The new method is analogous to smoothing via diffusion, and its implementation requires only a discrete Laplacian operator appropriate to the data. The new method can approximate arbitrary filter shapes, including Gaussian filters, and can be extended to spatially varying and anisotropic filters. The new diffusion‐based smoother's properties are illustrated with examples from ocean model data and ocean observational products. An open‐source Python package implementing this algorithm, called gcm‐filters, is currently under development.